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FOREWORD 


This  research  and  development  was  conducted  in  support  of  task  area 
ZF57.525.00 1.023.03.02  (Mathematical  Models  of  Manning  Utilization).  The  overall 
objective  of  this  task  area  was  to  develop  and  demonstrate  improved  methodologies  for 
assessing  variations  in  manpower  utilization  on  the  operational  performance  of  Navy 
platforms. 

This  report  documents  the  capabilities  and  shortfalls  of  the  SHIP-II  model,  which  was 
developed  to  assess  the  adequacy  of  manning  suites  for  Navy  ships.  It  was  determined 
that  further  development  of  SHIP-II  is  not  warranted  because  of  cost  considerations  and 
the  difficulties  involved  in  simulating  shipboard  operations  as  they  affect  manpower 
utilization  and  scheduling.  Since  this  report  describes  some  of  these  difficulties,  it  should 
be  useful  for  future  research  on  similar  undertakings. 

Acknowledgements  are  due  to  S.  E.  Bowser,  E.  R.  N.  Robinson,  R.  N.  Harris,  and  P. 
Billingsley  of  the  Navy  Personnel  Research  and  Development  Center  for  their  early 
contributions  to  the  documentation  of  the  original  SHIP-II  model.  Acknowledgements  are 
also  due  to  the  project  personnel  of  B-K  Dynamics,  San  Diego,  California,  who  performed 
much  of  the  analysis  of  SHIP-II. 
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SUMMARY 


Problem 


The  SHIP-II  computer  simulation  model  was  developed  to  assess  the  adequacy  of 
manning  suites  for  Navy  ships.  The  model  has  been  extensively  revised  since  its  initial 
development  in  1962,  but  no  extensive  verification  or  validation  of  the  model  had  been 
conducted. 

Objectives 

The  objectives  of  this  effort  were  to  verify  and  validate  the  SHIP-II  model  and 
determine  its  utility  for  assessing  manpower  requirements  for  Navy  ships. 

Approach 

The  computer  program  was  verified  by  a  careful  examination  of  the  program  logic  to 
ensure  that  it  does  what  it  was  intended  to  do.  A  baseline  stability  analysis  and  a 
sensitivity  analysis  were  also  conducted  to  determine  model  response  to  changes  in  the 
input  variables.  Model  validation  was  conducted  by  critically  examining  the  underlying 
model  assumptions  to  determine  if  they  might  cause  the  model  to  diverge  from  reality. 
An  effort  was  made  to  compare  actual  man-hour  data  gathered  from  USS  ARTHUR  W. 
RADFORD  (DD  968)  with  results  of  a  SHIP-II  simulation  of  this  ship,  using  as  the  scenario 
the  actual  operation  scenario  experienced  by  RADFORD  during  the  time  period  that  man¬ 
hour  data  was  collected.  An  exercise  was  also  conducted  to  compare  the  results  of 
changing  the  manning  suite  for  a  typical  DD  963.  Two  manning  suites  were  used— manning 
taken  from  the  Ship  Manning  Document  (SMD)  and  manning  from  the  Navy  Manning  Plan 
(NMP).  A  post-processor  was  built  to  facilitate  the  statistical  analysis  of  the  outputs 
f'om  a  simulation  run. 

Findings  and  Discussion 

1.  The  SHIP-II  model  verification  analysis  and  manning-suite  change  exercise  show 
that  the  existing  computer  logic  and  programs  do  not  accurately  model  the  desired  ship, 
equipment,  and  manpower  interactions.  Many  logic  deficiencies  and  omissions  detract 
from  its  fidelity. 

2.  The  baseline  stability  analysis  and  sensitivity  analysis  show  that  the  model 
exhibits  the  desired  stability  over  time  and  the  expected  response  to  changes  in  the  input 
variables.  However,  the  degree  of  this  response  could  not  be  measured  due  to  lack  of:  (a) 
enough  data  to  make  a  statistical  statement  about  the  changes  (due  to  excessive  computer 
costs  required  to  generate  the  data)  and  (b)  confidence  in  the  fidelity  of  the  model. 

3.  While  the  results  of  the  comparison  of  model  output  and  actual  data  are 
generally  favorable,  the  results  of  the  model  validation  experiment  must  be  considered 
inconclusive  due  to  the  limitations  of  the  actual  data  collected. 

4.  The  use  of  the  model  is  limited  because  data  input  requirements  are  excessively 
high.  For  example,  to  exercise  one  ship  in  the  model  requires  approximately  3  man- 
months  to  acquire  and  assemble  data  not  in  a  machine-readable  format. 


Conclusions 


The  SHIP-n  simulation  model  is  not  an  adequate  tool  for  examining  manpower 
requirements  and  manning  policies  for  Navy  ships.  It  is,  however,  the  only  model  that 
encompasses  all  ship  evolutions  and  an  operational  scenario.  There  are  several  logic 
deficiencies  and  omissions  that  must  be  rectified  to  ensure  fidelity  of  the  model.  These 
required  changes,  together  with  excessive  and  complex  data  requirements,  preclude  the 
use  of  SH1P-II  by  Navy  operational  managers. 

Recommendation 


Because  of  cost  considerations  and  the  technical  difficulties  involved  in  developing  a 
useful  ship  simulation  model,  further  efforts  should  not  be  made  to  modify  and  further 
develop  SHIP-II  at  this  time. 
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INTRODUCTION 


Problem 


The  SHIP-11  model  was  developed  to  provide  a  simulation  of  a  Navy  ship  for  the 
purpose  of  studying  the  interaction  of  manpower,  ship's  equipment,  ship's  operational 
employment,  and  personnel  and  management  policies  and  constraints.  While  there  is  a 
need  in  the  Navy  for  manpower  analysis  techniques  such  as  the  SHIP-II  mode!,  their 
technical  adequacy  must  be  carefully  examined.  A  computer  simulation  is  only  as  sound 
as  the  techniques  and  data  used  in  constructing  it.  Prior  to  its  use,  a  simulation  model 
must  be  validated,  or  shown  to  actually  model  the  system  being  studied.  Research 
applications  of  computer  simulation  models  and  the  decisions  based  on  such  applications 
have  often  preceded  objective  evaluations  of  the  models  involved.  Although  several 
small-scale  special-purpose  studies  conducted  at  various  stages  of  the  model's  develop¬ 
ment  have  tended  to  confirm  the  validity  of  SHIP-11,  it  has  never  been  subjected  to  a  full- 
scale  verification  and  validation  effort. 

Objectives 


The  objectives  of  this  effort  were  to  verify  and  validate  the  SHIP-II  model  and 
determine  its  utility  for  assessing  manpower  requirements  for  Navy  ships. 

Background 

SHIP-II  is  designed  to  simulate  normal  activities  conducted  aboard  a  Navy  ship.  The 
simulation  model  is  used  to  determine  the  impact  of  various  operational  and  workload 
factors  on  personnel  resources.  SHIP-II  is  an  event-oriented,  digital  model  that  uses  the 
Monte  Carlo  method  to  produce  random  samples  of  events  from  empirically-derived 
frequency  distributions.  Events  include  the  assignment  of  personnel  to  watches,  perfor¬ 
mance  of  maintenance  and  work,  equipment  failures,  training  exercises  and  classes,  and 
other  normal  activities  and  occurrences  on  a  Navy  ship.  The  model  is  driven  by  a  scenario 
that  lists  initiation  times  and  durations  for  operational  events  such  as  training  exercises, 
changes  in  readiness  conditions,  and  evolutions. 

SHIP-II  was  developed  as  a  follow-on  to  the  DDG  2  Total  Ship  Simulation  Model. 
Since  1970,  the  SHIP-II  model  has  undergone  several  major  revisions.  Most  revisions  are 
due  to  attempts  to  apply  the  model  to  a  variety  of  manpower  and  personnel  research 
problems.  The  result  is  a  large-scale  computer  simulation  model  of  a  destroyer-class  ship. 
This  analysis  concentrated  on  the  verification  and  validation  of  the  SHIP-II  model  in  its 
present  form. 


APPROACH 

A  simulation  model  is  validated  by  demonstrating  that  it  is  an  accurate  representa¬ 
tion  of  the  real  system  it  is  designed  to  simulate.  Validation  should  not  be  confused  with 
verification.  Verification  of  a  computer  program  is  an  examination  of  the  program  logic 
and  output  to  ensure  that  it  does  what  it  was  intended  to  do.  A  verified  computer 
program  can,  in  fact,  represent  an  invalid  model.  That  is,  the  program  logic  may 
accomplish  exactly  what  the  programmer  intended  without  representing  the  operation  of 
the  real  system. 

Validation  of  computer  simulation  models  is  a  difficult  task  and  there  is  little 
agreement  about  what  constitutes  adequate  validation.  Despite  this  lack  of  agreement, 
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there  are  techniques  that  have  proven  useful  in  model  validation.1  One  technique  is  to 
compare  the  results  of  the  simulation  with  results  historically  produced  by  the  real  system 
operating  under  the  same  conditions.  A  second  technique  is  to  use  the  simulator  to 
predict  results  that  can  then  be  compared  with  the  results  produced  by  the  real  system 
during  some  future  period  of  time.  Hidden,  but  nonetheless  critical,  assumptions  may 
cause  the  model  to  diverge  from  reality.  One  purpose  of  the  model  validation  process  is 
to  discover  such  assumptions.  They  are  best  detected  by  a  diligent  examination  of  the 
model's  underlying  assumptions  and  design. 

The  analysis  included  in  this  effort  consisted  of:  (1)  a  review  of  existing  SHIP-II 
documentation  (a  rough  draft  of  the  user's  guide)2  and  a  careful  examination  of  the 
computer  code,  (2)  a  sensitivity  and  stability  analysis  of  the  model's  output  parameters, 
(3)  a  comparison  of  actual  man-hour  data  gathered  from  USS  ARTHUR  W.  RADFORD  (DD 
968)  with  results  of  a  SHIP-II  simulation  of  this  ship,  and  (4)  a  comparison  of  SHIP-II 
output  for  two  distinct  manning  suites  for  a  typical  DD  963  class  ship.  The  sensitivity  and 
stability  analyses  revealed  major  flaws  in  the  model.  During  this  investigation,  even 
thoug  results  indicated  that  stability  and  a  degree  of  sensitivity  exist  in  the  output 
parameters,  the  output  levels  of  several  parameters  appeared  suspicious.  This  prompted 
careful  examination  of  the  computer  code,  which,  in  turn,  led  to  the  discovery  of  several 
flaws  in  the  model's  logic.  Many  other  problems  were  encountered  during  the  subsequent 
comparison  of  SHIP-II  output  for  two  distinct  manning  suites. 


FINDINGS  AND  DISCUSSION 

This  description  of  the  analysis  performed  on  SHIP-II  is  divided  into  two  major 
categories.  The  first  area  concerns  the  results  of  all  efforts  categorized  as  a  verification 
analysis.  These  include  (1)  a  careful  examination  of  the  actual  computer  code  to 
determine  how  accurately  the  model  reflects  current  Navy  ship  tasking  practices,  (2)  a 
test  for  the  stability  of  the  model  results  by  using  different  random  number  seeds  to  begin 
a  simulation,  and  (3)  an  analysis  to  test  the  output  variable  sensitivities  to  changes  in  the 
input  parameters  and  to  determine  the  workload  impact  of  varying  the  manning  levels  and 
skills  provided  as  the  ship's  crew. 

The  second  type  of  analysis  (usually  called  validation)  consisted  of  two  major  efforts. 
First,  in  conjunction  with  the  verification,  several  special  runs  of  the  model  were  made 
utilizing  output  commands  to  determine  the  fidelity  of  the  model's  watch  and  mainte¬ 
nance  assignments  and  to  check  that  man-hours  are  accumulated  in  each  activity 
correctly  (a  face  validity  test).  Finally,  an  attempt  was  made  to  compare  results  of  a 
SHIP-II  simulation  of  RADFORD  with  man-hour  data  that  was  collected  by  project 
personnel  while  aboard  the  same  ship. 


i  ‘For  an  exposition  of  this  problem,  see  Schechter,  M.  and  Luca,  R.  C.  Validating  a 

large-scale  simulation  model  of  wilderness  recreational  travel.  Institute  of  Management 
Sciences,  1980,  1_0(5),  11. 

2A  copy  of  this  guide  may  be  obtained  upon  request  from  NAVPERSRANDCEN,  Code 


11. 


Verification  Analysis 

Code  Examination  and  a  Face  Validity  Test 

Face  validity  is  evaluated  in  terms  of  the  "reasonableness"  of  a  simulation  model's 
output.  To  this  end,  several  runs  of  the  SHIP-11  model  were  made,  over  periods  of  21  to  35 
days  (using  an  input  data  set  developed  for  a  typical  DD  963).  The  model  was  instructed 
to  give  a  snapshot  of  its  transaction  table  and  work  assignments  every  4  hours  of 
simulation  time.  Concurrent  scrutiny  of  the  output  and  its  associated  input,  as  well  as 
careful  examination  of  the  underlying  computer  code  comprising  the  various  subroutines 
of  SHIP-II,  revealed  a  number  of  model  logic  deficiencies  and  omissions  that  detract  from 
the  fidelity  of  SHIP-11.  The  problems  encountered  are  discussed  below. 

No  Fair-Sharing  of  Workload.  The  earlier  documentation  of  division  training  states 
that  individuals  are  assigned  to  a  class  from  the  top  of  the  division  manning  list  downward 
until  the  class  is  filled.  Because  of  this,  individuals  near  the  bottom  of  the  list  might  not 
accumulate  as  many  division  training  hours  as  would  those  nearer  the  top  of  the  list.  The 
same  logic  is  used  in  the  assignments  to  critical  maintenance  and  other  work.  Conse¬ 
quently,  the  work  is  not  fair-shared,  which  usually  results  in  a  wide  disparity  of  the 
average  weekly  workloads  for  men  in  the  same  division.  The  men  near  the  top  of  the 
division  list  tend  to  have  workloads  that  exceed  the  Navy  standard  workweek,  while  those 
near  the  bottom  tend  to  have  workloads  that  fall  short  of  a  standard  workweek.  In  some 
cases,  the  difference  is  as  much  as  50  man-hours/week. 

The  SHIP-II  program  contains  a  subroutine,  LAZFST,  which  sorts  a  list  of  man- 
numbers  by  hours  worked  and  puts  the  man  with  the  least  hours  of  work  at  the  top  of  the 
list  before  scheduling  work.  However,  this  subroutine  is  only  called  from  two  places  in 
the  program.  In  both  instances,  the  jobs  being  scheduled  are  administrative  support. 
There  is  no  apparent  reason  why  it  is  not  used  for  other  work  scheduling. 

The  following  example  illustrates  this  non-fair-sharing  of  work.  Table  1  shows  the 
work  assignments  of  a  GSM3  NEC  4111  (man  No.  11  in  the  MP  Division)  over  a  4-day 
period.  Table  2  shows  the  number  of  hours  three  identical  men  in  the  MP  Division  worked 
during  the  same  4-day  period. 

Each  man  is  a  GSM3  NEC  4111  and  each  stands  watch.  The  difference  in  the  total 
man-hours  worked  by  the  four  men  is  caused  by  the  repeated  corrective  maintenance  (CM) 
requirements  placed  on  the  first  man.  When  total  workload  for  this  rate  and/or  NEC 
(Navy  enlisted  classification)  code  is  examined,  this  may  not  be  a  problem  because 
averages  over  the  total  ouput  are  used.  However,  there  may  be  a  distortion  in  the  man¬ 
hours  worked  by  a  particular  NEC  when  the  model  assigns  work  by  rate  only  when  there  is 
more  than  one  NEC. 

Incomplete  Transaction  not  Appearing  in  the  Output.  In  the  previous  example,  an 
individual  apparently  worked  25  hours  in  a  24-hour  day  because  of  a  transaction  that 
began  on  one  day  and  did  not  end  until  the  following  day.  All  hours  spent  on  this 
transaction  are  attributed  to  the  second  24-hour  day.  In  practice,  the  errors  are  small. 
Nonetheless,  a  user  should  be  aware  that  minor  errors  might  occur  from  week  to  week. 

Inaccurate  Preventive  Maintenance  (PM)  Scheduling.  Based  on  a  statistical  analysis 
of  the  data  generated  from  five  different  runs  of  SHIP-II  with  different  random  number 
seeds  (this  will  be  further  discussed  later  in  this  report),  the  hypothesis  that  there  is  no 
increase  or  decrease  in  PM  man-hours  over  time  must  be  rejected.  Each  of  the  five 
baseline  runs  showed  a  statistically  sign  if  it  ant  increasing  trend  in  PM  man-hours  over  the 
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Table  1 


Assignment  of  Man  No.  1 1  Over  4-day  Period 


Time 

Monday 

Tuesday 

Wednesday 

Thursday 

Total 

(Hrs) 

0100 

Watch 

Watch 

CM 

Watch 

0500 

Idle 

CM 

Watch 

CM 

0900 

PM 

CM 

CM 

CM 

1300 

Watch 

Watch 

CM 

Watch 

1700 

Watch 

Watch 

Watch 

Idle 

2100 

CM 

CM 

CM 

Watch 

Total  (Hrs)  19.72 

19.00 

25.46a 

19.00 

83.18 

Average 

20.80 

Note.  CM  is  corrective  maintenance  and  PM,  preventive  maintenance. 

aThe  accumulated  total  is  greater  than  24.0  hours  because,  when  a  maintenance  action  is 
continued  past  2400  on  one  day,  the  accumulated  task-hours  are  added  to  the  following 
day. 


Table  2 

Assignment  of  Men  Nos.  12-14  Over  4-day  Period 


Man 

Hours  Per  Day 

Monday 

Tuesday 

Wednesday 

Thursday 

Total 

12 

14.75 

19.89 

19.00 

23.86 

77.50 

13 

14.75 

17.50 

17.49 

16.50 

66.24 

14 

18.75 

22.28 

13.75 

65.28 

13-week  scenario.  The  reason  for  this  trend  is  believed  to  be  in  the  way  the  model 
simulates  the  performance  of  PM.  The  model's  treatment  of  PM  appears  to  be  both 
inaccurate  and  unrealistic.  The  causes  can  be  attributed  to  both  the  PM  input  data  and 
the  program  logic.  Some  specific  faults  with  the  PM  logic  are: 

1.  The  PM  workload  (measured  in  man-hours)  increases  over  time,  because  of  (a) 
the  steady  buildup  from  week  to  week  of  unaccomplished  PM  carried  over  to  the  next 
week  and  (b)  the  scheduling  of  semi-annual,  annual,  and  cyclic  PM  tasks  late  in  the 
scenario. 


2.  When  a  PM  check  cannot  be  conducted  because  of  other  actions  (higher  priority 
work,  equipment  failure,  etc.),  the  model  attempts  to  reschedule  the  check  every  45 
minutes.  This  is  an  inefficient,  possibly  excessive,  use  of  computer  time  to  accomplish 
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PM.  The  frequency  of  rescheduling  attempts  is  not  in  consonance  with  common  Navy 
practice. 

The  net  effects  of  these  faults  are  important  to  the  operation  of  the  model.  The 
model,  which  presents  an  unrealistically  pessimistic  view  of  the  PM  workload,  has  a 
reasonable  scheme  for  scheduling  part  of  the  PM.  Daily  and  weekly  checks  are  scheduled 
every  day  and  on  the  first  day  of  the  week,  respectively.  The  monthly,  quarterly,  semi¬ 
annual,  annual,  and  cyclic  checks  are  evenly  distributed  within  each  period.  For  example, 
if  16  monthly  checks  are  on  the  list,  four  are  scheduled  each  week  of  the  month. 
However,  the  semi-annual,  annual,  and  cyclic  checks  do  not  begin  to  appear  on  the 
schedule  until  the  latter  portion  of  the  run.  Once  the  model  begins  to  run,  any 
interruptions  or  delays  in  scheduled  PM  (and  there  are  many)  cause  the  PM  workload  to 
begin  a  steady  growth  as  the  PM  tasks  are  rescheduled  in  the  following  days  and  weeks. 

There  are  four  changes  that  could  be  made  to  improve  the  PM  logic: 

1.  Extend  the  at-sea  PM  workday  from  the  current  0800  to  1630  to  0800  to  2200. 

2.  Rather  than  attempting  to  schedule  PM  at  the  beginning  of  each  day  and 
rescheduling  every  45  minutes,  schedule  PM  following  watch  assignments  and  changes  in 
readiness  conditions  from  Condition  I  or  II  to  Condition  III  or  IV. 

3.  Allow  PM  to  be  performed  on  watch. 

4.  Give  PM  a  higher  priority  relative  to  other  workload  factors. 

Deferrals  for  Assistance.  Deferrals  for  assistance  are  made  when  the  repair  of  the 
existing  casualty  requires  the  material  resources  of  a  repair  facility  or  the  assistance  of 
specially  trained  and  experienced  maintenance  specialists,  who  are  normally  in  scarce 
supply  and  stationed  ashore.  This  outside  repair  assistance  is  generally  not  available  until 
the  ship  next  enters  port.  In  the  current  model,  the  deferral  for  assistance  is  executed  at 
the  time  the  decision  is  made  to  defer  and  remains  in  effect  for  some  time  specified  in 
the  input  data  set.  It  would  be  more  realistic  to  simulate  the  decisicn  to  defer,  as  is 
currently  done,  and  then  delay  the  completion  of  this  deferred  maintenance  action  until 
the  next  in-port  period.  The  affected  item  of  equipment  would  remain  out  of  service  for 
the  remainder  of  the  current  at-sea  period  and  for  the  expected  repair  time  period  after 
reaching  port. 

Watchstanding  Policy.  As  a  general  rule,  a  man  on  watch  would  remain  on  watch 
until  a  suitable  relief  is  provided.  If  the  normal  relief  is  not  available  due  to  other 
commitments,  then  the  watchstander  should  remain  at  his  watch  beyond  his  normal  hours. 
In  the  current  model,  if  the  normal  relief  is  not  available  and  a  substitute  cannot  be 
bumped  from  other  requirements,  the  model  will  terminate.  Contemporary  Navy  practice 
would  have  the  current  watchstander  remain  at  his  post  until  a  suitable  relief  can  be 
located  and  assigned. 

Evolutions  versus  Equipment  Readiness.  As  a  general  rule,  even  the  most  important 
evolutions  would  be  postponed,  or  cancelled,  if  certain  subsystems  or  equipment  were 
inoperable.  For  example,  a  ship  would  not  perform  an  underway  replenishment  alongside 
if  any  part  of  the  steering  system  were  out  of  service.  No  attempt  would  be  made  to 
anchor  if  the  anchor  windlass  were  defective.  While  the  model  checks  for  necessary 
subsystem  and  equipment  readiness  before  attempting  to  perform  training  exercises,  it 
does  not  follow  this  practice  for  evolutions.  There  were  cases,  for  example,  where  the 
model  was  underway  and  performing  evolutions  with  the  main  propulsion  system  out  of 
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service  and  rated  Condition  IV.  Before  attempting  to  perform  a  training  exercise,  the 
model  first  checks  the  status  of  each  subsystem  and  equipment  critical  to  the  exercise.  If 
they  are  not  in  an  acceptable  state  of  readiness,  the  exercise  is  not  conducted.  This 
concept  should  be  applied  to  evolutions. 

Repair  Team  Size.  Whenever  a  team  is  required  to  perform  a  maintenance  action, 
the  model  limits  the  team  size  to  four  men.  This  has  apparently  not  been  a  limiting 
factor  in  the  ships  tested  thus  far  in  SHIP-II.  However,  some  ships  (particularly 
replenishment  ships)  require  large  teams  to  perform  some  routine  PM.  Also,  some  new 
PM  concepts  such  as  total  ship  test  plan  require  large  teams  for  routine  PM.  Additionally, 
the  model's  team  requirement  for  maintenance  is  too  rigid.  If  a  team  cannot  be 
assembled,  the  model  cancels  or  delays  the  work.  In  Navy  practice,  this  would  not 
happen. 

In-port  Duty  Section  Assignments.  The  in-port  work  schedule  and  workload  of  a  ship's 
crew  are  greatly  affected  by  the  number  of  in-port  duty  sections.  The  current  Navy  goal 
is  to  place  crews  in  six  in-port  duty  sections.  However,  the  model  provides  for  only  four 
in-port  sections  and  only  one  watch  assignment  per  section.  Moreover,  the  number  of 
duty  sections  for  watch  are  all  fixed  for  each  simulation  run.  They  should,  if  realistic,  be 
allowed  to  vary,  depending  on  the  operational  scenario. 

Number  of  Maintenance  Tasks.  The  number  of  PM,  facilities  maintenance  (FM),  and 
administrative  support  (S/A)  tasks  is  limited  by  the  model's  size.  As  a  result,  the  original 
SHIP-II  designers  increased  the  magnitude  of  each  individual  task  in  an  attempt  to  enter 
the  entire  tasking  workload.  Consequently,  some  tasks  are  not  performed  because  they 
require  too  large  a  block  of  time. 

"As  required"  PM.  A  concept  analogous  to  the  previous  discussion  of  the  relationship 
between  equipment  readiness  and  evolutions  is  the  "as  required"  PM.  In  a  typical  ship,  20 
to  30  percent  of  the  PM  requirements  are  not  linked  to  any  specific  periodicity;  rather, 
they  are  performed  when  certain  situations  arise.  For  example,  a  number  of  PM  checks 
are  performed  on  the  guns  and  ammunition  handling  equipment  prior  to  a  gun-shoot.  If 
the  checks  are  not  satisfactory,  the  shoot  is  not  conducted.  As  the  model  does  not 
consider  this  type  of  PM,  it  understates  the  actual  PM  man-hours  required.  This  problem, 
along  with  several  of  the  problems  discussed  above,  force  the  nonwatchstanding  workload 
to  be  much  lower  than  would  be  realistically  expected. 

Bumping  Logic.  The  model's  bumping  logic  for  job  priorities  (see  Appendix  A)  is 
basically  correct  for  the  long  run.  However,  it  is  fixed  throughout  the  simulation  run  and 
should  be  allowed  to  vary  based  on  the  operational  scenario. 

Nonwatchstanding  tasks.  Another  problem  concerns  the  modeling  of  nonwatchstand¬ 
ing  tasks.  The  model  does  have  an  "up"  option  on  a  task  allowing  the  user  to  specify  that 
a  job  should  be  performed  by  a  specific  rating-rate.  However,  if  this  man  is  busy,  it  will 
"up"  to  the  next  rate  above  and  assign  this  man  the  task  if  he  is  not  busy,  and  so  on,  up  to, 
at  most,  3  "ups."  This  is  realistic,  but  the  model  is  not  sensitive  to  different  repair  times 
experienced  by  more  senior  personnel  in  each  rating.  There  is  probably  a  significant 
difference  here. 

Readiness.  The  model  scores  readiness,  based  on  the  material  readiness  portion  of 
the  Navy  force  operational  readiness  statistics  (NAVFORSTAT)-defined  readiness 
decisions.  It  does  not  consider  supply,  training,  or  skilled  personnel  constraints,  as  does 
the  new  UNITREP  system.  This  leads  to  an  overly  optimistic  score  of  ship  readiness. 
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Troubleshooting  Time.  The  time  spent  troubleshooting  a  failure  has  been  eliminated 
from  the  model.  Although  the  Navy's  3-M  Maintenance  Data  System  (MDS)  collects  man¬ 
hour  data  on  corrective  maintenance  (CM),  it  does  not  distinguish  mean  time  in  function 
troubleshooting  (MTTS).  Moreover,  the  probability  that  troubleshooting  is  required  is 
reported  on  the  casualty  reports  (CASREPs)  only  for  mission-critical  equipments.  Thus, 
the  troubleshooting  feature  of  the  model  has  been  eliminated.  However,  experience  as 
well  as  expert  testimony  has  revealed  that  a  great  deal  of  time  is  spent  in  this 
maintenance  category.  Again,  SHIP-11  does  not  account  for  some  of  the  nonwatchstanding 
workload. 

Model  Data  Requirements.  A  number  of  the  model's  input  data  requirements  appear 
to  have  been  formulated  based  on  information  that  was  to  have  been  available  in  the 
Navy's  3-M  MDS  when  the  model  was  originally  developed.  In  its  initial  development,  the 
3-M  MDS  was  to  have  collected  extensive  data  about  shipboard  PM  and  CM.  However, 
after  an  adverse  reaction  from  the  fleet,  the  maintenance  data  collection  requirements 
were  reduced  significantly.  Today,  the  3-M  MDS  does  not  collect  any  data  on  PM 
activities.  The  only  available  data  on  PM  man-hours  are  the  engineered  standards  listed 
on  the  PM  requirement  cards  (PMCs).  The  MDS  collects  data  on  only  a  small  portion  of 
the  total  CM  activities  and,  of  that,  only  the  total  current  maintenance  man-hours  are 
useful.  No  troubleshoot  time  is  available.  Current  SHIP-11  data  requirements  cannot  be 
met  because  of  these  changes  in  maintenance  data  collection. 

Stability  of  Model 

Five  baseline  stability  runs  were  made  to  test  the  degree  of  consistency  in  the  model. 
Each  run  used  the  DD  963  class  input  data  base,  a  13-week  scenario,  and  a  different 
random  number  seed.  The  details  of  this  analysis  are  discussed  in  Appendix  B. 

The  results  of  the  analysis  among  runs  using  different  random  number  seeds  support 
the  hypothesis  that  changing  the  random  number  seed  in  otherwise  identical  runs  of  the 
model  does  not  change  the  underlying  population  parameters.  There  were  no  statistically 
significant  differences  among  the  outputs  of  SH1P-II  runs  employing  different  random 
number  seeds.  All  the  runs  appear  to  be  random  samples  taken  from  the  same  population. 

In  an  effort  to  detect  trends  in  the  model  output  over  time,  the  means  and  variances 
of  each  variable  were  computed  over  all  runs.  Simple  linear  regression  was  performed  on 
these  values  to  discern  possible  trends.  The  mathematical  details  ere  also  found  in 
Appendix  B.  Trend  analysis  results  showed  some  variation  in  the  model  output  over  time 
in  some  cases.  In  particular,  it  was  found  that  PM  increased  over  time  in  all  cases.  This 
problem  as  well  as  its  resolution  were  mentioned  earlier  in  the  results  of  the  verification 
analysis. 

Sensitivity  Analysis 

The  sensitivity  of  the  model  to  changes  in  independent  or  input  variables  was  tested 
with  repeated  simulation  runs.  In  each  run,  all  input  variables,  except  one,  were  held  at 
baseline  levels.  The  exception  in  each  run  was  a  specific  change  in  one  of  the  input 
variables  listed  below: 

1.  Mean  time  between  failures  (MTBF). 

2.  Mean  time  in  function  repair  (MTFR). 

3.  Mean  time  in  function  PM  (MTFPM). 

4.  Probability  of  deferrals  for  assistance. 

5.  Mean  time  of  deferrals  for  assistance. 


6.  Probability  of  deferrals  for  parts. 

7.  Mean  time  of  deferrals  for  parts. 

The  mean  and  variance  of  each  output  variable  of  interest  were  computed  and  used  to 
test  the  hypothesis  that  the  sample  mean  value  resulting  from  a  sensitivity  run  was  equal 
to  the  baseline  mean  value  of  that  variable.  The  model  is  sensitive  to  changes  in  the  input 
variables  when  this  hypothesis  can  be  rejected.  The  output  data  and  statistical 
computations  of  the  sensitivity  analysis  runs  are  displayed  in  Appendix  B. 

The  results  of  the  sensitivity  analysis  runs,  while  generally  favorable,  were  not  as 
definitive  as  the  baseline  stability  runs.  In  general,  the  model  reacts  to  changes  in  input 
parameters  in  expected  ways.  For  example,  a  decrease  in  equipment  MTBF  causes  an 
expected  increase  in  man-hours  expended  in  CM.  The  limited  number  of  sensitivity 
analysis  runs  for  each  variable  does  not  provide  a  large  enough  sample  to  permit 
predicting  the  range  of  output  values  resulting  from  specific  changes  to  an  input  variable. 
The  specific  results  achieved  with  changing  some  of  the  above  input  parameters  are 
described  below. 

1.  Mean  Time  Between  Failures  (MTBF).  Four  runs  of  the  model  were  made  using 
random  number  seed  177.  In  the  four  runs,  the  parameter  variation  ratio  (PVR)  for  MTBF 
was  set  to  make  changes  in  MTBF  of  -90,  -70,  -50,  and  -10  percent.  Statistically 
significant  changes  in  CM  man-hours  and  total  man-hours  were  seen  in  response  to  the 
-50,  -70,  and  -90  percent  changes.  Without  extensive  additional  runs  of  the  model,  it  is 
not  possible  to  determine  the  exact  relationship  between  variations  in  MTBF  and  CM  man¬ 
hours.  For  example,  using  seed  177,  when  MTBF  was  reduced  to  10  percent  of  its  baseline 
value,  the  model  generated  5  times  as  many  failures  for  the  main  propulsion  gas  turbines 
and  6.28  times  as  many  CM  man-hours  overall.  Neither  of  these  values  are  predictable 
from  analysis  of  existing  data. 

2.  Mean  Time  in  Function  Repair  (MTFR).  Three  model  runs  were  made  to  test 
sensitivity  of  changes  in  time  in  function  for  equipment  repair.  Again,  random  number 
seed  177  was  used.  The  PVR  for  MTFR  was  set  to  make  changes  of  +90,  +50,  and  +30 
percent  and  the  resultant  changes  in  CM  man-hours  were  recorded  and  analyzed.  As  with 
MTBF,  the  changes  were  in  the  expected  directions. 

3.  Mean  Time  in  Function  PM  (MTFPM).  Thiee  runs  of  the  model  were  made,  using 
random  number  seed  177,  to  test  the  sensitivity  of  changes  in  MTFPM.  The  PVR  for 
MTFPM  was  set  to  increase  the  input  variable  MTFPM  by  +90,  +50,  and  -r30  percent,  and 
the  resultant  changes  in  PM  man-hours  were  recorded  and  analyzed.  As  in  the  case  of 
variations  in  MTBF,  increases  in  MTFPM  caused  the  expected  corresponding  increases  in 
PM  man-hours. 

4.  Deferrals  for  Parts  and  Assistance  (Covers  5  and  7  above).  Runs  of  the  model 
were  made  to  test  sensitivity  to  changes  in  input  parameters  controlling  deferrals. 
Changes  in  deferral  for  parts  probability,  mean,  and  standard  deviation  were  made  (10%, 
0.01  hr,  and  0.01  hr  were  changed  to  25%,  120  hrs,  and  48  hrs,  respectively).  Results  were 
similar  to  those  seen  in  changes  to  MTBF,  MTFPM,  and  MTFR.  That  is,  the  parameter 
changes  caused  expected  changes  in  output  but  of  unpredictable  amounts.  The  number  of 
deferrals  and  the  average  time  per  deferral  both  increased. 

The  sensitivity  of  the  model's  readiness  ranking  system  was  tested  in  conjunction  with 
changes  in  the  MTFR  parameter.  When  MTFR  was  increased  by  90  percent,  the  output 
readiness  states  were  analyzed  to  determine  if  longer  repair  time  would  cause  a  reduction 
in  the  average  time  spent  in  readiness  condition  Cl  and  an  increase  in  time  spent  at  lower 
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conditions  of  readiness  (CII--CIV).  Table  3  shows  the  results  of  this  test.  The  expected 
results  were  realized.  A  comparison  study  was  made  of  the  percentage  of  total  scenario 
time  spent  in  readiness  condition  Cl  versus  the  time  spent  in  conditions  C1I--CIV  on  the 
baseline  run  and  on  the  run  where  MTFR  was  increased  by  +90  percent.  The  analysis 
indicated  a  definite  shift  toward  the  lower  readiness  conditions  when  MTFR  was 
increased.  However,  as  experienced  in  all  previous  analyses,  the  specific  degree  of 
change  could  not  be  predicted  from  the  available  data. 


Table  3 

Readiness  Summary 


Week 

Baseline 

MTFR  Increased  by  90% 

Time  at  Cl 
(%) 

Time  at  CII,  III,  IV 
(%) 

Time  at  Cl 
(%) 

Time  at  CII,  III,  IV 
(%) 

1 

95.6 

4.4 

69.6 

30.4 

2 

78.2 

21.8 

42.0 

58.0 

3 

58.2 

41.8 

68.0 

32.0 

4 

95.8 

4.2 

37.0 

63.0 

5 

84.2 

15.8 

1.6 

98.4 

6 

85.8 

14.2 

40.1 

59.9 

7 

69.8 

30.2 

42.7 

57.3 

8 

90.3 

9.7 

0.0 

100.0 

9 

79.4 

20.6 

89.8 

10.2 

10 

69.5 

30.5 

26.1 

73.9 

11 

71.2 

28.8 

48.7 

51.3 

12 

99.5 

0.5 

98.6 

1.4 

13 

31.2 

68.8 

36.4 

63.6 

Average 

77.6 

22.4 

46.2 

53.8 

Validation  Analysis 

Empirical  Validation  with  DP  968  Data 

Divisional  time-card  data  reflecting  the  actual  employment  of  ship's  personnel  in 
three  divisions  of  RADFORD  were  collected  by  Navy  personnel  and  provided  for  analysis 
and  comparison  with  the  output  of  SH1P-II.  These  data  were  collected  over  a  3-week 
period  during  the  deployment  of  the  ship  in  the  Mediterranean.  Personnel  of  the  auxiliary 
(A),  main  propulsion  (MP),  and  gunnery  (G)  divisions  recorded  their  employment  and  the 
time  spent  in  each  task.  The  ship's  logs  were  used  to  develop  a  6-month  employment 
scenario  for  use  in  the  SHIP-11  model  that  would  reflect  the  actual  employment  of  the  ship 
during  this  period. 

The  ship  was  at  sea  in  only  1  week  during  the  data  collection  effort.  The  majority  of 
in-port  time  was  spent  on  holiday  routine  (half  workdays)  and  maximum  liberty  for  the 
crew.  The  SHIP-II  model  is  not  presently  programmed  to  permit  changes  in  the  desired 
workweek  during  model  runs.  Given  the  limited  data  actually  collected  (3  weeks  for  the  A 
and  G  divisions  and  2  weeks  for  the  MP  Division)  and  the  difference  between  the 
workweek  of  the  model  (standard  Navy  workweek)  and  the  ship  in-port  (holiday  routine),  a 


close  comparison  of  model  predictions  and  the  actual  historical  data  could  not  be 
accomplished.  The  results  of  the  comparison  must  be  considered  inconclusive. 

Navy  Manning  Plan/Ship  Manning  Document  Alternative  Manning  Analysis 

To  determine  the  sensitivity  of  SHIP-II  to  alternative  manning  suites,  two  52-week 
runs  of  the  model  were  made  by  varying  only  the  input  data  set  by  the  manning  suite.  The 
first  run  used  a  manning  suite  extracted  from  the  Ship  Manning  Document  (SMD)  for  the 
DD  963.  This  document  is  prepared  jointly  by  the  Navy  Manpower  and  Material  Analysis 
Center,  Atlantic  (NAVMMACLANT)  and  the  Navy  Manpower  and  Material  Analysis 
Center,  Pacific  (NAVMMACPAC)  for  each  class  of  Navy  ship  in  the  fleet.  It  is  based 
mostly  on  recorded  data  of  work  performed  on  the  ships  (i.e.,  it  is  ship-task  generated  and 
is  not  constrained  by  the  number  of  bunks  available  on  the  ship  or  by  the  pool  of  man¬ 
power  available  to  the  Navy). 

The  second  run  of  SH1P-II  was  made  with  the  same  set  of  input  data,  except  for  the 
manning  suites.  This  time  the  manning  suite  was  extracted  from  the  Navy  Manning  Plan 
(NMP).  This  manning  suite  is  constructed  basically  from  the  SMD  after  it  has  passed 
through  the  appropriate  Navy  channels  for  approval.  During  this  process,  personnel  are 
removed  from  the  original  SMD  to  account  for  bunk  constraints  and  to  ensure  a  fair¬ 
sharing  of  the  personnel  pool.  Appendix  C  illustrates  the  differences  between  the  two 
manning  suites  defined  for  the  DD  963. 

Two  realizations  were  made  from  this  exercise.  First,  the  work  input  for  each 
category  of  labor  (kept  the  same  for  each  run  and  consisting  of  tasks  defined  by  the  SMD) 
was  essentially  accomplished  by  both  manning  suites.  However,  the  NMP  manning 
required  heavier  workloads  from  its  personnel  than  the  SMD  did.  Moreover,  the  number 
and  length  of  scheduled  and  unscheduled  maintenance  queues  (due  to  the  lack  of  available 
qualified  personnel)  was  greater  for  the  NMP  than  the  SMD.  Second,  SHIP-II  also  scores 
the  ship's  readiness  (based  solely  on  equipment  availability).  It  was  found  that  both 
manning  suites  produced  roughly  the  same  ship  readiness  score  or,  more  specifically,  the 
same  overall  mission  critical  equipment  availability. 

It  may  be  concluded  that  the  problems  discussed  earlier  concerning  the  model  logic 
and  computer  code  have  distorted  the  true  maintenance  and  readiness  picture.  Otherwise, 
the  nonwatchstanding  tasks  defined  by  the  SMD  are  an  understatement  of  the  work  that 
can  be  accomplished  by  this  manning  suite.  Several  selected  weekly  report  summaries, 
which  are  displayed  in  Table  4,  illustrate  the  differences  in  the  results  using  the  two 
manning  suites.  These  weeks  were  selected  depending  on  whether  the  ship  was  at  sea  or 
in-port  during  the  entire  week.  Reports  were  selected  at  the  beginning,  middle,  and  end 
of  the  52-week  scenario.  A  few  divisions  under  the  NMP  manning  suite  were  severely 
overworked.  The  primary  cause  for  the  overworking  can  be  attributed  to  the  fact  that 
SHIP-II  did  not  prevent  personnel  who  had  already  put  in  a  full  Navy  workday  from 
working. 

The  final  observation  made  regarding  the  alternative  manning  exercise  concerns  the 
amenability  of  SHIP-II  for  performing  this  type  of  analysis.  It  is  very  difficult  and  time 
consuming  to  use  SHIP-II  for  making  alternative  manning  studies  of  a  given  ship 
class/type.  If  the  researcher  chooses  to  begin  the  analysis  with  a  baseline  manning  suite 
and  then  alters  the  same  baseline  suite,  the  task  is  conceivable.  However,  if  the 
researcher  attempts  to  compare  the  results  of  two  totally  disjoint  manning  suites,  the 
task  approaches  almost  impracticability.  This  is  attributable,  primarily,  to  the  input  data 
requirements.  If  the  two  manning  suites  are  very  disjoint,  as  in  the  case  of  the  NMP  and 
SMD,  the  divisions  are  usually  misaligned.  This  necessitates  careful  and  tedious 


Selected  Weekly  Results  of  Alternative  Manning  Exercise 


Note.  All  upper  values  are  for  NMP  manning  and  lower  values  are  for  SMD  manning. 


reorganization  of  the  input  data  set,  particularly  the  watch  station  assignments,  division 
definitions,  and  secondary  NEC  assignments.  Consequently,  a  large  number  of  input  data 
cards  must  be  changed.  The  current  input  data  set  for  SHIP-II  has  many  sections  of  data 
that  are  dependent  on  each  other  (i.e.,  a  change  made  in  one  section  results  in  many 
similar  changes  made  in  other  sections).  If  all  the  resultant  changes  are  not  also  made, 
SHIP-11  will  not  complete  processing  but  will  terminate  immediately. 

Still  another  problem  encountered  in  using  SHIP-II  concerns  the  computer  time  and 
output  statistical  analysis  required.  In  order  to  perform  any  output  data  analysis 
correctly,  it  is  necessary  to  make  multiple  runs  of  SHIP-II  varying  the  random  number 
seed.  This  is  extremely  expensive  in  terms  of  computer  time  and  results  in  a  large 
amount  of  data  that  a  team  of  analysts  must  reduce  in  order  to  perform  the  statistical 
analysis.  It  should  be  noted  that  the  results  presented  in  Table  4  do  not  reflect  this  type 
of  complete  analysis;  rather,  they  were  generated  by  one  run  of  SHIP-II  with  only  one 
random  number  seed. 


CONCLUSIONS 

The  SHIP-II  simulation  model  is  a  deficient  method  for  examining  manpower 
requirements  and  manning  policies  for  Navy  ships.  This  report  identifies  certain  logic 
deficiencies  and  omissions  in  the  model  that  detract  from  its  fidelity.  Moreover,  because 
of  the  many  intricacies  of  the  model's  input  and  output  and  the  general  logistics  of  any 
analysis  employing  SHIP-II  (such  as  the  SMD/NMP  manning  suite  analysis  performed  in 
this  effort),  a  "quick-and-dirty"  probe  into  a  manning  problem  cannot  be  made  with  this 
model. 

To  make  the  model  useful,  it  would  be  necessary  to  restructure  the  model's  input. 
Currently,  the  SHIP-II  input  requirements  are  not  only  massive  but  extremely  complex. 
The  various  model  input  sections  are  interdependent  and  yet  have  distinct  and  very 
different  Navy  data  sources.  Consequently,  much  effort  is  required  to  make  the  input 
sections  consistent.  All  of  this  work  must  now  be  accomplished  manually. 


RECOMMENDATION 

Because  of  cost  considerations  and  the  technical  difficulties  involved  in  developing  a 
useful  ship  simulation  model,  further  efforts  should  not  be  made  to  modify  and  develop 
the  SHIP-II  model  at  this  time. 
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BUMPING 


The  logic  of  bumping  is  based  upon  the  following  matrix,  which  is  found  in  the  1MITIL 
subroutine. 
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At  each  point  in 

the  simulation, 

every 

man 

has  e 

l  status  based  on  his  job. 

The 

definition  of  each  status  number  is: 

0  Idle 

1  Facilities  Maintenance  (FM) 

2  Administrative  Support  (S/A) 

3  Division  Training 

4  Preventive  Maintenance  (PM) 

5  Corrective  Maintenance  (noncritical)  (CM) 

6  CM  (critical) 

7  Train  Exercise 

8  Watch  (repair) 

9  Watch  (substitute) 

10  Watch  (regular) 

1 1  Evolution 

The  rationale  for  the  number  codes  in  the  matrix  is  explained  in  the  following 

statements: 

•  Generally:  a  5  gives  an  error;  a  1  allows  a  bump;  a  2  indicates  no  bump  and  no 
error;  and  a  3  allows  a  bump. 

•  Watch  (10)  bumps  any  column  with  a  1  or  3  in  it.  A  5  causes  an  error. 

•  When  a  repair  team  is  being  assembled  for  a  job,  a  1  in  the  column  for  the  new 
job  allows  a  bump. 

•  An  evolution  (11)  will  interrupt  any  job  with  a  1  or  3.  A  5  causes  an  error. 

•  For  a  training  exercise  (7),  a  5  causes  an  error,  a  I  or  3  can  be  interrupted. 

•  When  a  substitute  watch  is  being  assembled,  column  10  is  examined  for  a  1. 


Occasionally,  a  status  comparison  may  be  made  in  the  program  without  going  through 
the  matrix.  However,  the  checks  observed  in  the  test  run  seem  to  conform  to  the  matrix. 

The  documentation  of  the  current  model  states  that  critical  equipment  is  maintained 
over  all  other  jobs.  However,  that  is  false.  Watch,  training  exercises,  and  evolutions  are 
all  more  important  than  critical  equipment  repair. 


APPENDIX  B 

VERIFICATION  AND  VALIDATION  STATISTICS 


Page 

Baseline  Stability  Analysis . B- 1 

Sensitivity  Analysis . B-21 


B-0 


VERIFICATION  AND  VALIDATION  STATISTICS 


Baseline  Stability  Analysis 

Model  reliability  can  be  defined  as  the  degree  of  consistency  in  the  model;  that  is, 
the  degree  to  which  the  same  model,  operating  on  the  same  inputs,  will  produce  consistent 
outputs.  In  SHIP-11,  random  events  are  determined  by  a  series  of  pseudorandom  numbers 
drawn  from  a  random  number  generator.  These  pseudorandom  numbers  meet  certain 
criteria  for  randomness,  but  always  begin  with  a  certain  initial  value  called  the  seed  and 
proceed  in  a  completely  deterministic,  repeatable  fashion.  SHIP-II  simulation  runs  with  a 
fixed  input  and  the  same  random  number  seed  will  produce  identical  outputs.  The  choice 
of  a  different  random  number  seed  for  SHIP-II  will  produce  a  different  output.  The  first 
and  most  important  objective  of  the  baseline  stability  analysis  is  to  determine  if  this 
apparent  difference  resulting  from  different  random  number  seeds  represents  random 
samples  drawn  from  the  same  population  or  some  real  difference  in  the  model. 

Once  started,  the  SHIP-II  model  will  simulate  ship  operations  over  a  predetermined 
length  of  time,  typically  13  to  26  weeks.  Model  consistency  over  time  would  require  that 
the  selected  output  variables  such  as  watch  hours,  divisional  training  hours,  corrective 
maintenance  man-hours,  and  planned  maintenance  man-hours  be  independent  of  time. 

If  the  model  meets  these  tests  of  consistency,  then  the  five  baseline  runs  can  be 
considered  random  samples  drawn  from  the  same  population.  The  output  data  of  these 
runs  can  then  be  used  to  estimate  the  population  parameters  for  the  output  variables  of 
interest. 

Baseline  Stability  Tests 

To  test  whether  a  change  in  the  random  number  seed  would  result  in  statistically 
significant  changes  in  the  output  variables  for  a  fixed  input,  a  single  classification  or  one¬ 
way  analysis  of  variance  (ANOVA)  was  performed. 

In  this  design,  each  run  using  a  different  random  number  seed  can  be  considered  to 
represent  one  and  only  one  of  five  distinct  populations  with  means  ul,  y2,  .  .  .  y5.  We 
will  test  the  hypothesis  that  all  five  means  are  equal. 

For  each  output  variable  of  interest,  the  13  weekly  values  in  a  particular  simulation 
run  can  be  considered  to  be  a  random  sample  drawn  from  one  of  the  five  populations. 
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Analysis  Procedure 

The  experimental  design  is: 

Random  number  seed 
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where  n.  is  the  number  of  measurements  in  each  population  (n  =  13  in  this  case). 
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where  N 


2  Mr 

i  =  1 
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The  population  variance  will  be  estimated  in  two  ways  and  compared.  A  pooled 
2 

estimate  of  a  can  be  computed  from 


22 


2  ",  - K 

where  K  is  the  number  of  distinct  categories  or  populations  (K  =  5  in  this  case). 

The  variance  of  the  means  can  be  computed  from 

T  2  T2 

i+  _ ++ 

2  ni  N 

S  1  =  - - - 

m  K  -  1 

We  wish  to  reject  the  hypothesis  of  no  difference  in  means,  if  the  observed  means  are 
significantly  more  disperse  than  we  would  expect  when  all  obtained  from  the  same 
population. 

2  2 

The  two  quantities,  S  and  S  ,  may  be  tested  for  significant  difference  by  using 
the  F  ratio.  pm 


The  hypothesis  of  equal  means  would  be  rejected  when  the  F  ratio,  Fg,  obtained  exceeds 
the  critical  value  of  the  F  table  for  K-i  and  ” 

2  m  -  K  degrees  of  freedom  at  the  chosen  level  of  significance. 

An  ANOVA  table  can  be  used  to  summarize  the  results  of  these  computations. 


Means 


Within 


Sum  of  Squares 
2  2 
V  Ill  .  Lti 

n.  N 

22  V  -2  Hr 


Degrees  of  Mean 

Freedom  Square 


Total 
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The  hypothesis  of  no  difference  in  categories  (equal  means)  will  be  tested  at  the  5- 
percent  level  of  significance  and  it  will  be  accepted  if  the  observed  F  value  is  less  than 
F  95  (4.60)  =  2.53. 

Trend  Analysis 

Model  consistency  over  time  requires  that  the  values  of  the  output  variables  be 
independent  of  time.  One  criterion  of  independence  is  that  the  mean  of  this  variable  be 
the  same  for  each  increment  of  time. 

In  the  case  of  trend  analysis  over  time,  this  means  that  the  mean  of  the  dependent 
variable  Y  is  the  same  for  each  increment  of  time  X  or  that  the  slope  of  the  trend  line 
b  =  0. 

The  equation  of  a  trend  line  is 
Y=  a  +  b  X 


where:  Y  is  the  estimated  value  of  the  dependent  variable, 

X  is  the  independent  variable  time, 
a  is  the  Y  intercept  (the  value  of  Y  when  X  =  0),  and 
b  is  the  slope  of  the  trend  line. 


2 

S7 


n  X  Y 


a  =  Y  -  b  X 


2x 


To  test  for  independence,  we  will  test  the  hypothesis  that  the  slope  b  =  0.  If  this 
hypothesis  cannot  be  rejected,  we  shall  conclude  that  there  is  sufficient  reason  to  believe, 
at  the  specified  level  of  significance,  that  Y  is  independent  of  X.  > 


The  test  statistic  is 


where  S  = . 
e 


2y2  -  a^Y  -  b2: 


The  hypothesis  that  b  =  0,  Y  is  independent  of  X,  will  be  accepted  if 


where  t  =  2.201  at  the  5  percent  level  of  significance  and  11  degrees  of  freedom. 

Analysis  Results 

The  results  of  the  baseline  stability  analysis  for  each  of  the  variables  of  interest  are 
presented  on  pages  B-6  through  B-20.  Each  page  includes: 

1.  The  output  data  for  each  random  number  seed  with  the  mean  and  standard 
deviation  shown  for  each  run. 

2.  The  ANOVA  table  used  to  test  the  hypothesis  of  equal  means  between  runs  for 
this  variable. 

3.  The  mean  value  of  this  variable  of  interest  over  all  runs  together  with  its 
standard  deviation  and  the  95  percent  confidence  limits  about  the  mean. 

k.  The  results  of  the  trend  analysis  performed  for  each  run.  The  slope  and 

2 

intercept  are  displayed  together  with  the  coefficient  of  regression,  r  ,  and  the  test 
statistic,  ts. 
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PREVENTIVE  MAINTENANCE  MAN-HOURS 


s:f.o 
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WEEK  1 
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13 
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GROUP 

MEAN 

196.18 

195.94 

203.17 

202.83 

182.72 

S.D. 

43.198 

43.426 

50.723 

43.203 

52.838 

SUMMARY  TABLE 

ANALYSIS  OF  VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

135361 

1 

BETWEEN 

3564.31 

4 

891.078 

WITHIN 

131796 

60 

2196.61 

F-RATIO 

=  .405661  <  2 

.53 

DEGREES  OF 

FREEDOM 

=  4  &  60 

BASELINE 

MEAN  199 

.44  mnhrs 

S.D.  43 

.790  mnhrs 

95%  Conf  limits  188.80  <  X<  210.09 

TREND  ANALYSIS 

SEED 

129 

157 

177 

199 

999 

SLOPE 

7.10 

6.40 

7.50 

7.25 

7.53 

intercept 

1A5.51 

152.13 

150.68 

152.06 

145.39 

.r2 

0.41 

0.33 

0.33 

0.43 

0.43 

ts 

2.761 

2.326 

2.336 

2.856 

2.395 
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WATCH  MANHOURS 
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1776.00 

1776.00 

10 

1812.00 

1812.00 

1811.76 

1798.19 

1810.63 
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MEAN 
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S.D. 
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328.028 
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ANALYSIS  OF  VARIANCE 
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SS 

DF 

MS 

TOTAL 
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4 

39 
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6.44379E+06 

60 

107396 

F-RATIO 
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DEGREES  OF  FREEDOM  =  4  &  60 
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MEAN 
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S.D. 
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CORRECTIVE  MAINTENANCE  MANHOURS 
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35.89 
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26.02 

35.92 

32.50 

13.78 

29.46 

6 

49.75 

19.86 

33.20 

12.30 

8.35 

7 

44.99 

93.43 

67.98 

15.17 

115.73 

8 

55.91 

121.00 

29.69 

63.02 

73.00 

9 

77.38 

30.80 

53.85 

8.10 

123.03 

10 

3.93 

39.32 

83.99 

117.14 

33.29 

11 

4.16 

128.12 

102.87 

100.18 

64.34 

12 

14.73 

29.90 

2.59 

79.05 

30.76 

13 

16.30 

109.42 

109.00 

21.69 

61.32 

GROUP 

MEAN 

61.01 

55.89 

61.91 

51.74 

56.40 

S.D. 

55.355 

41.218 

37.28 

35.999 

35.274 

SUMMARY 

TABLE 

_ANALYSIS  OFJ/ARIANCE 


SOURCE  SS  DF  MS 


TOTAL 

105959 

1 

BETWEEN 

603.547 

4 

150.887 

WITHIN 

105365 

60 

1755.09 

F-RATIO 

= 

.0859222  <  2.5 

DEGREES 

OF  FREEDOM 

4  &  60 

BASELINE 

MEAN 

57.39 

mnhrs 

S.D. 

40.546 

mnhrs 

93%  Conf  limits 

47.54  < 

J  <  67.25 

TREND  ANALYSIS 

SEED 
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999 

SLOPE 
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*> 
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0.02 

0.02 
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-3.022 

1 . 931 

0.270 

0.467 

0.439 

B— 8 


TOTAL  MANHOURS 


1. 


2. 


SEED 

129 

1SZ 

in 

199 

m 

'WEEK  1 

23bj .36 

2822.23 

2319.55 

2342.83 

2319.34 

2 

2363.79 

2805.44 

2837.37 

2839.84 

2841.33 

3 

2421.73 

2310.67 

2453.23 

2351.80 

2354.89 

4 

3173.80 

3037.33 

3050.01 

3061.03 

3004.16 

5 

2930.50 

2953.32 

2942.20 

2935.68 

2923.78 

6 

2214.29 

2146.70 

2160.48 

2138.85 

2147.26 

7 

2771.37 

2836.06 

2789.51 

2757.66 

2847.85 

8 

2859.51 

2912.94 

2820.21 

2867.38 

2870.01 

9 

2910.46 

2880.34 

2952.30 

2888.89 

2993.51 

10 

2847.15 

2897.30 

2932.59 

2944.69 

2866.86 

11 

2376.42 

2496.39 

2486.04 

2513.41 

2461.87 

12 

2077.89 

2110.74 

2076.52 

2160.77 

2103.19 

13 

2856.90 

2927.54 

2993.92 

2924.07 

2912.71 

GROUP 

MEAN 

2704.4 

2702.16 

2716.46 

2709.77 

2703.29 

S.D. 

323.249 

322.509 

319.36 

311.372 

320.499 

SUMMARY  TA8LE 

ANALYSIS  OF  VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

6.07514E+06 

1 

BETWEEN 

3408 

4 

852 

WITHIN 

6.07173E+06 

60 

101195 

3. 


4. 


F-RATIO 

= 

8.41935E-03  <  2.53 

DEGREES  OF  FREEDOM 

= 

4  &  60 

BASELINE 

MEAN 

2707.35  mnhrs 

S.D 

309.168  mnhrs 

95%  Conf 

limits 

2632.19  <  X  < 

2782.51 

TREND  ANALYSIS 

SEED 

129 

157 

177 

199  999 

SLOPE 

-24.53 

-10.69 

12.63 

-10.86  -12.43 

INTERCEPT  2376.10  2776.91 


r* 

ts 


0.09 

-1.026 


0-02 

-0.431 


2804.88 

0.02 

-0.517 


2785.76 

0.02 

-0.455 


2790.93 

0.02 

-0.508 


DIVISION  TRAINING  MANHOURS 


3. 


SSIQ 

129 

157 

177 

199 

999 

V1"K 

1 

20.53 

20.50 

20.50 

19.50 

20.50 

2 

22.50 

22.50 

22.50 

21.00 

21 . 03 

3 

19.50 

20.50 

37.43 

19.00 

20.00 

4 

28.03 

29.00 

28.50 

20.50 

30.53 

5 

20.50 

20.50 

20.50 

20.50 

21.50 

6 

31.00 

19.50 

18.50 

19.50 

20.00 

7 

18.50 

18.50 

18.50 

19.50 

19.00 

8 

20.50 

18.50 

19.50 

38.40 

18.50 

9 

29.50 

31.00 

20.50 

30.00 

19.50 

10 

18.50 

20.50 

18.00 

19.50 

18.50 

11 

18.50 

29.50 

29.00 

40.87 

19.50 

12 

19.00 

19.00 

19.00 

26.50 

18.00 

13 

22.00 

19.00 

29.50 

29.08 

19.50 

GROUP 

MEAN 

22.19 

22.19 

23.23 

24.91 

20.46 

S.D. 

4.395 

4.507 

5.978 

7.576 

3.173 

SUMMARY  TABLE 

ANALYSIS  OF  VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

1852.6 

64 

BETWEEN 

138.282 

4 

34. 

5706 

WITHIi 

H 

1714.32 

60 

28. 

5719 

F-RATIO 

=  1 

.20995 

DEGREES 

OF  FREEDOM 

=  4 

&  60 

BASELINE 

MEAN 

22.60  tnnhrs 

S.D 

5.380  mnhrs 

952  Conf  1 

imits  21.29 

<  X  <  23.90 

TREND 

i  ANALYSIS 

SEED 

129 

157 

177 

199 

999 

SLOPE- 

-0.13 

0.22 

-0.15 

1.14 

-0. 

35 

INTERCEPT 

23.48 

22.04 

24.29 

16.94 

22. 

94 

r* 

0.03 

0.00 

0.01 

0.34 

0. 

19 

ts 

-0.543 

0.063 

0.331 

2.397 

-1. 

599 

B-10 


SUPPORT/ADMINISTRATIVE  MANHOURS 


SEED 

129 

157 

177 

199 

991 

WEEK  1 

187.13 

187.97 

187.79 

187.64 

188.00 

2 

191.81 

191.81 

191.81 

191.81 

191.81 

3 

205.68 

205.68 

205.68 

205.68 

205.68 

4 

183.10 

183.93 

187.57 

183.78 

183.78 

5 

185.20 

185.50 

185.51 

185.51 

185.21 

6 

214.89 

215.29 

215.29 

215.29 

215.29 

7 

204.24 

204.27 

204.27 

203.47 

206 

8 

217.22 

216.92 

217.32 

216.70 

21 

9 

178.25 

191.41 

191.90 

184.00 

ISi 

10 

207.24 

207.09 

207.14 

207.24 

207.  i-r 

11 

162.15 

161.75 

162.15 

162.15 

162.15 

12 

201.37 

201.37 

201.37 

200.97 

201.77 

13 

185.99 

191.59 

216.23 

240.67 

186.21 

GROUP 

MEAN 

194.18 

195.74 

198.00 

198.84  . 

195.95 

S.D. 

15.867 

14.926 

16.620 

19.639 

15.085 

SUMMARY  TABLE 

ANALYSIS  OF  VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

16164 

64 

BETWEEN 

182.438 

4 

45.6094 

WITHIN 

15981.6 

60 

266.36 

=  .171232 
=  4  &  60 


TREND  ANALYSIS 


SEED  129 

157 

177 

199 

999 

SLOPE  '  -0.40 

-0.13 

0.65 

1.42 

-0.20 

IN  fECEPT  196.95 

196.64 

193.47 

188.89 

197.35 

r2  0.01 

0.001 

0.03 

0.08 

0.003 

ts  -0.325 

-0.111 

0.543 

0.974 

-0.172 

F-RATIO 

DEGREES  OF  FREEDOM 


BASELINE 

MEAN  196.54  mnhrs 

S.D.  15.891  mnhrs 

95%  Conf  Limits  192.68  <  X  <  200.41 


B-ll 


FACILITIES  MAINTENANCE  MANHOURS 


SEED 

lii 

157 

177 

199 

999 

wee:<  i 

437.27 

488.73 

488.53 

490.00 

487.05 

2 

511.02 

514.97 

513.01 

512.23 

514.07 

3 

499.82 

501.92 

498.85 

502.38 

592.  >’8 

4 

525.16 

535.68 

513.66 

519.20 

513.02 

5 

505.89 

505.42 

506.39 

506.32 

506.13 

6 

495.64 

495.47 

495.52 

495.12 

495.47 

7 

507.52 

507.92 

510.14 

509.61 

509.61 

8 

509.28 

505.13 

508.25 

508.54 

509.14 

9 

481.35 

470.51 

475.90 

490.52 

490.25 

10 

500.76 

500.28 

504.60 

501.01 

500.83 

11 

462.71 

468.04 

464.08 

467.22 

468.20 

12 

494.46 

494.33 

494.23 

494.96 

494.60 

13 

514.34 

518.16 

519.50 

504.90 

529.92 

GROUP 

MEAN 

499.63 

500.58 

499.44 

500.16 

501.63 

S.D. 

16.045 

13.327 

15.887 

13.146 

15.213 

SUMMARY  TABLE 
ANALYSIS  OF  VARIANCE 


SOURCE 

SS 

OF 

MS 

TOTAL 

15028.7 

64 

BETWEEN 

29 

4 

7.25 

WITHIN 

14999.7 

60 

249.995 

F-RATIO  =  .0290006 


DEGREES  OF  FREEDOM  =  4  &  60 


BASELINE  MEAN 
S.D. 

95%  Conf  Limits 

500.24 
15.214 
495.54  . 

mnhrs 
mnhrs 
<  X  <  503 

.94 

TREND  ANALYSIS 

SEED 

129 

157 

177 

199 

999 

SLOPE  ■ 

-0.93 

-1.25 

-0.67 

-1.14 

-0.18 

INTERCEPT 

505.46 

509.33 

504.15 

507.90 

502.90 

r- 

0.05 

0.07 

0.03 

0.11 

0.002 

ts 

-0.803 

-0.914 

-0.556 

-1.179 

-0.159 

B-12 


SEED 

129 

157 

122 

199 

999 

WEEK 

1 

123.95 

154.82 

160.60 

148.23 

120.94 

2 

127.54 

134.03 

131.37 

78.42 

ill. 89 

3 

143.00 

149.57 

97.83 

143.91 

149.55 

4 

64.49 

114.26 

160.89 

154.50 

163.00 

5 

161.76 

123.76 

141.45 

152.21 

164.75 

6 

29.90 

154.40 

144.21 

168.00 

166.65 

7 

136.14 

54.55 

117.32 

165.14 

125.69 

8 

168.00 

'  33.90 

151.66 

119.40 

75.67 

9 

98.97 

146.85 

133.37 

163.01 

91.63 

10 

155.67 

138.85 

116.73 

85.62 

154.57 

11 

166.44 

94.26 

119.54 

94.02 

137.00 

12 

162.75 

144.58 

167.15 

152.15 

168.00 

13 

151.71 

53.04 

52.39 

152.34 

132.41 

GROUP 

MEAN 

131.56 

115.14 

130.35 

136.69 

135.90 

S.O. 

43.01 

42.554 

31.07 

31.402 

33-109 

3. 


SUMMARY  TABLE 

ANALYSIS  OF 

VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

78821.6 

64 

BETWEEN 

5337.63 

4 

1334.41 

WITHIN 

73484 

60 

1224.73 

F-RATIO 

=  1.08955 

DEGREES 

OF  FREEDOM 

=  4  &  60 

BASELINE  • 

MEAN  131.47 

mnh  rs 

S.D.  33.545 

S'6%  Conf  Limits  123.31 

<  X  <  139 

.62 

TREND  ANALYSIS 

SLED 

129 

157 

177 

199 

999 

SLOPE 

3.31 

-4.28 

-2.88 

-0.22 

0.12 

INTERCEPT 

115.09 

145.13 

150.53 

138.22 

135.07 

r‘ 

is 


1.551 


-1.413 


-1.285 


-0.090 


0.059 


B-13 


■** 


HOURS -TOTAL 

SHIP  READINESS  SCORE  ( 

.  -2 

SEED 

129 

157 

177 

159 

999 

Vi  lE  c_  «\ 

1 

39.06 

13 . 19 

1.30 

12.57 

47.0/ 

2 

3.60 

28.73 

27.02 

89.58 

6.12 

3 

2.89 

13.01 

2.23 

24.11 

13.45 

4 

1.94 

41.92 

2.45 

9.80 

0 

5 

1.35 

17.65 

20.34 

12.02 

3.25 

6 

0 

13.60 

22.91 

0 

0 

7 

7.65 

97.22 

27.58 

0 

42.32 

8 

0 

24.58 

13.51 

13.89 

53.26 

9 

0 

18.04 

24.10 

0 

21.41 

10 

0 

26.35 

51.27 

.29 

3.04 

11 

1.56 

21.38 

1.65 

10.57 

31.00 

12 

5.25 

2.48 

.85 

15.86 

0 

13 

9.50 

21.54 

66.35 

7.95 

35.59 

GROUP 

MEAN 

5.6 

26.13 

20.89 

15.13 

20.50 

S.D. 

10.503 

23.362 

20.854 

23.540 

20.455 

SUMMARY 

TABLE 

ANALYSIS  OF 

VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

27909 

.3 

64 

BETWE 

EN 

3147. 

49 

4 

786.873 

WITHIN 

24761 

.8 

60 

412.697 

F- RAT 10 

=  1.90666 

DEGREES 

OF  FREEDOM 

=  4  &  60 

BASELINE  MEAN  17.65  mnhrs 
S.D.  20.83  mnhrs 
9b%  Corif  Limits  12.57  £  X  £  22.73 


4.  TREND  ANALYSIS 


SEED 

123 

157 

177 

199 

9'?  3 

SLOPE  . 

-1.01 

-0.45 

2.21 

-2.69 

0 . 30 

INTERCEPT 

12.64 

29.31 

5.44 

33.94 

18.41 

r2 

0.14 

0.01 

0.17 

0.20 

.003 

ts 

-1.322 

-0.252 

1.500 

-1.646 

U.139 

fl-14 


liQURS-‘f OTAL  SHIP  READINESS  SCORE  C-3 


SEED 

129 

1S7 

177 

199 

939 

V.'EEK  1 

0.03 

0.00 

6.11 

7.22 

0 

2 

36.86 

5.24 

9.61 

0 

50.00 

3 

17.11 

0.00 

67.94 

0 

0 

4 

91.78 

11.83 

4.66 

3.70 

0 

5 

4.89 

26.63 

6.21 

3.76 

0 

6 

38.09 

0.00 

0.89 

0 

1.35 

7 

19.29 

15.96 

.92 

2.68 

0 

8 

0.00 

0.00 

2.10 

34.70 

34.07 

9 

69.03 

3.11 

10.53 

4.99 

54.96 

10 

2.33 

2.78 

0.00 

82.08 

10.40 

11 

0.00 

52.36 

45.49 

63.42 

0 

12 

0.00 

20.93 

0.00 

0 

0 

13 

6.79 

93.43 

49.26 

7.72 

0 

GROUP 

MEAN 

22.01 

19.09 

15.67 

17.29 

11.70 

S.D. 

29.494 

26.728 

22.778 

26.387 

20.439 

SUMMARY 

TABLE 

ANALYSIS  OF  VARIANCE 

^  _  _  _  _ 

SOURCE 

SS 

DF 

MS 

TOTAL 

39376.6 

1 

BETWEEN 

771.101 

4 

192.775 

WITHIN 

38605.5 

60 

643.425 

F-RATIO 

=  .299608  < 

2.53 

DEGREES  OF  FREEDOM 

4  &  60 

BASELINE 

MEAN 

16.66  mnhrs 

S.D. 

25.279  mnhrs 

95%  Conf 

Limits 

10.58  <  X  <  22 

.75 

4.  TREND  ANALYSIS 


SEED 

129 

157 

177 

199 

939 

SLOPE 

-2.14 

4.25 

.64 

2.91 

-.42 

INTERCEPT 

37.02 

-11.91 

11.17 

-4.17 

14.53 

r2 

0.08 

0.37 

0.01 

0.18 

0-01 

ts 

-0.979 

2.541 

0.366 

1.533 

-  .265 

B-15 


HOURS— TOTAL  SHIP  READINESS  SCORE  C-4 


SEED 

m. 

O0 

157 

177 

199 

999 

WEEK  1 

0T00 

O0 

T 

0 

2 

0.00 

0.00 

0.00 

0 

0 

3 

0.00 

5.42 

0-00 

0 

0 

4 

9.79 

0.00 

0.00 

0 

0 

5 

0.00 

0.00 

0-00 

0 

0 

6 

0.00 

0.00 

0-00 

0 

0 

7 

4.92 

0.28 

12.18 

.19 

0 

8 

0.00 

109.53 

0.72 

0 

0 

9 

0.00 

0.00 

0.00 

0 

0 

10 

0.00 

0.00 

0.00 

0 

0 

11 

0.00 

0.00 

1.33 

0 

0 

12 

0.00 

0.00 

0.00 

0 

0 

13 

0-00 

0.00  ' 

0.00 

0 

0 

GROUP 

MEAN 

1.13 

8.86 

1.09 

0.01 

0 

S.D. 

2.936 

30.283 

3.355 

0.053 

0 

SUMMARY 

TABLE 

_  _  _  _  __  _ 

ANALYSIS  OF 

VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

11976 

.4 

1 

BETWEEN 

732.994 

4 

183.248 

WITHIN 

11243 

.4 

60 

187.39 

F-RATIO  =  .977899<  2.53 

DEGREES  OF  FREEDOM  =  4  &  60 


BASELINE 

MEAN 

2.22  mnhrs 

S.D. 

13.679  mnhrs 

95%  Conf 

Limits 

-1.10  <  X  <  5.55 

TREND  ANALYSIS 


SEED 

129 

157 

177 

199 

999 

SLOPE 

-0.16 

0.48 

0-03 

0.00 

-.02 

INTERCEPT 

2.26 

5.49 

0.86 

0.01 

0.27 

r2 

0.05 

0.003 

0.001 

0.01 

0.15 

ts 

-0.727 

0.206 

0.128 

0.00 

-1.38? 

B-16 


HOURS- PROPULSION  SUBSYSTEM  READINESS  SCORE  C-l 


SEED 

129 

157 

177 

199 

999 

WEEK  1 

128.96 

168.02 

lb0 . 60 

166.70 

135.71 

2 

131.15 

137.88 

158.39 

168.00 

118.23 

3 

150.73 

159.95 

100.06 

154.68 

163.08 

4 

66.44 

152.33 

165.55 

154.50 

168.00 

5 

161.76 

139.90 

143.53 

159.90 

164.75 

6 

129.90 

168.00 

167.11 

168.00 

168.00 

7 

138.36 

102.50 

121.68 

165.14 

125.69 

8 

168.00 

33.90 

155.75 

123.57 

80.16 

9 

98.97 

168.00 

152.62 

163.01 

121.89 

10 

165.67 

160.46 

163.97 

98.33 

154.57 

11 

168.00 

161.17 

120.38 

104.67 

153.53 

12 

168.00 

147.07 

163.00 

168.00 

168.00 

13 

161.21 

53.02 

59.14 

160.29 

168.00 

GROUP 

KEAN 

141.32 

134.79 

141.29 

150.37 

145.35 

S.D. 

30.857 

44.441 

32.610 

24.692 

27.244 

SUMMARY  TABLE 

_  analysisjdf 

VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

65829 

.8 

1 

BETWEEN 

1719. 

78 

4 

429.945 

WITHIN 

64110 

60 

1068.5 

F-RATIO 

=  .402382<  2.53 

DEGREES  OF  FREEDOM 

=  4  &  60 

BASELINE 

MEAN 

142.62  mnhrs 

S.D. 

32.072  mnhrs 

95%  Conf 

L'imi  ts 

134.83  <  X  < 

150.42 

TREND  ANALYSIS 

SEED  129 

157 

177 

199 

999 

SLOPE 

3.61 

-3.81 

-2.62 

-2.45 

1.05 

IN rERCEPT 

116.05 

161.43 

159.65 

167.49 

1 3  8 .  jj  4 

r2 

0.21 

0.11 

0.10 

0.15 

0.02 

ts 

1.698 

-1.173 

-1.094 

-1.387 

0.50 

B-17 


■ill 


HOURS -PROPULS ION  READINESS  SCORE 

C-2 

SEED 

129 

157 

177 

199 

999 

WEEK  1 

39.06 

0.00 

1.30 

0 

3-09 

2 

0.00 

24.88 

0.00 

0 

6.12 

-i 

0.17 

8.05 

0.00 

13.33 

4.93 

4 

0.00 

3.84 

2.45 

4.80 

0 

5 

1.35 

1.47 

18.26 

4.34 

3.25 

6 

0.00 

0.00 

0.00 

0 

0 

7 

5.43 

65.21 

33.23 

0 

42.32 

8 

0.00 

24.58 

10.14 

9.72 

59.72 

9 

0.00 

0.00 

13.57 

0 

0 

10 

0.00 

7.39 

4.03 

0 

3.04 

11 

0.00 

0.00 

0.81 

0 

14.47 

12 

0.00 

0.00 

0.00 

0 

0 

13 

0.00 

21.55 

63.09 

0 

0 

GROUP 

MEAN 

■3.54 

12.07 

11.30 

2.86 

12.78 

S.D. 

10.779 

18.666 

18.417 

4.836 

19.510 

SUMMARY  TABLE 

ANALYSIS  OF  VARIANCE 

SOURCE 

SS 

DF 

MS 

TOTAL 

15733 

1 

BETWEEN 

1239.38 

4 

309.645 

WITHIN 

14493.6 

60 

241.559 

F- RAT 10 

= 

1.28269<  2.53 

DEGREES 

OF  FREEDOM 

= 

4  &  60 

BASELINE 

MEAN  8.51 

mnhrs 

S.D.  15.678  mnhrs 

95%  Conf 

Limits  4.70 

<  X  <  12.32 

TREND  ANALYSIS 

SEED  129 

157 

177 

199 

999 

slope 

-1.31 

0.03 

2.04 

-0.3;T 

('  C 

INTERCEPT 

12.68 

11.88 

-2.84 

5.66 

17.54 

r2 

.22 

0.00 

0.19 

0.05 

0.02 

ts 

-1.765 

0.019 

1.597 

-0.793 

-0  .45 
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HOURS  PROPULSION  SUBSYSTEM  READINESS  SCORE  C-3 


SEED 

129 

157 

177 

199 

999 

WEEK  1 

0 

0 

6.11 

1.32 

0 

2 

35.86 

5.24 

9.61 

0 

43.61 

3 

17.11 

0 

67.94 

0 

0 

4 

91.78 

11.83 

0 

3.70 

0 

5 

4.89 

26.63 

6.21 

3.76 

0 

6 

38.09 

0 

.89 

0 

0 

7 

19.29 

0 

.92 

2.68 

0 

8 

0 

0 

2.10 

34.70 

28.11 

9 

69.03 

0 

1.82 

4.99 

46.11 

10 

2.33 

.15 

0 

69.67 

10.40 

11 

0 

6.84 

46.82 

63.33 

0 

12 

0 

20.94 

0 

0 

0 

13 

6.79 

93.42 

45.77 

7.72 

0 

GROUP 

MEAN 

22.01 

12.70 

14.48 

14.76 

9.86 

S.D. 

29.494 

25.810 

23.015 

24.762 

17.468 

SUMMARY  TABLE 

_  _ _  _  _  _  _ 

ANALYSIS  OF  VARIANCE 

_ — 

SOURCE 

SS 

DF 

MS 

TOTAL 

36860.4 

64 

BETWEEN 

1051.94 

4 

262.986 

WITHIN 

35808.4 

60 

596.807 

F-RATIO 

=  .440654 

DEGREES 

OF  FREEDOM 

4  &  60 

3.  BASELINE  MEAN  14.76  mnhrs 

S.D.  23.999  mnhrs 

95%  Conf  Limits  8.92  <  X  <  20.60 


4.  TREND  ANALYSIS 


SEED 

129 

157 

177 

199 

999 

SLOPE 

-2.14 

3.18 

0.54 

2.89 

-0.3  V 

INTERCEPT 

37.02 

-9.54 

10.71 

-5.50 

12.42 

r2 

ts 

-0.979 

1.811 

0.303 

1.696 

-0.271 
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HOURS-PROPULSiON  SUBSYSTEM  READINESS  SCORE  C-4 


SPED 
lie  Ex  l 
2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 


GROUP 

MEAN 

S.D. 


■\  70 
0 
0 
0 

9.79 

0 

0 

4.92 

0 

0 

0 

0 

0 

0 


152 

D 

0 

0 

0 

0 

0 

.28 

109.53 

0 

0 

0 

0 

0 


1.13  8.45 

2.936  30.372 


.117 

0 

0 

0 

0 

0 

0 

12.18 

0 

0 

0 

0 

0 

0 


0.94 

3.378 


193 

a 

0 

0 

0 

0 

0 

.19 

0 

0 

0 

0 

0 

0 


0-01 

0.053 


999 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0-00 

0-00 


SUMMARY  TABLE 
ANALYSIS  OF  VARIANCE 


SOURCE  SS 


DF  MS  F-RATIO 


TOTAL 

11977 

64 

.88507 

BETWEEN 

667.327 

4 

166.832 

DEGREES  OF  FREEDOM 

WITHIN 

11309.7 

60 

188.495 

4  &  60 

BASELINE  MEAN  2.11  mnhrs 

S.D.  13.680  mnhrs 

95/i  Conf  Limits  -1.22  <  X  <  5.43 


TREND  ANALYSIS 

SEED 

129 

157 

SLOPE 

-0.16 

0.60 

INTERCEPT 

2.26 

4.23 

r2 

0-05 

0.01 

ts 

-0.727 

0.257 

177 

199 

999 

0.00 

0.00 

-0.03 

0.94 

0.01 

0.27 

0.00 

0.01 

0.15 

0.00 

0.00 

0.00 
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Sensitivity  Analysis 

Model  sensitivity  can  be  defined  as  the  degree  to  which  the  dependent  variable  of 
interest  is  responsive  to  changes  in  the  value  of  the  independent  variable  of  interest.  If 
the  output  variable  is  not  sensitive  to  the  value  of  the  input  variable,  then  either  the 
presumed  relationship  does  not  in  fact  pertain  or  the  relationship  is  masked  by  other 
effects  in  the  model.  If  the  model  is  valid,  then  the  lack  of  sensitivity,  mirroring  the  real 
world,  is  not  a  model  problem  but,  rather,  a  problem  in  the  choice  of  application.  A  valid 
model,  then,  is  useful  in  those  areas  in  which  it  is  sensitive. 

The  sensitivity  analysis  was  limited  to  the  smaller  data  base  used  in  the  baseline 
stability  tests.  The  main  propulsion  (MP)  division,  which  includes  both  main  propulsion 
and  auxiliary  systems  in  the  DD  963,  is  a  maintenance  intensive  division  that  should  show 
the  expected  response  to  maintenance  parameter  changes. 

The  sensitivity  analysis  consisted  of  the  following  steps: 

1.  Select  Dependent  Variables.  The  output  variables  examined  for  responsiveness 
to  changes  in  input  variables  which  were  the  same  as  those  selected  for  study  in  the 
baseline  stability  tests,  are  listed  below: 

a.  Watch  hours,  individual  and  total. 

b.  Divisional  training  hours. 

c.  Evolution  hours. 

d.  PM  hours. 

e.  CM  hours. 

f.  Ship's  work-hours. 

g.  Total  hours. 

2.  Select  Independent  Variables.  The  input  variables  selected  for  variation  in  the 
model  sensitivity  analysis  were: 

a.  Mean  time  between  failures  (MTBF). 

b.  Mean  time  in  function  repair  (MTFR). 

c.  Mean  time  in  function  preventive  maintenance  (MTFPM). 

d.  Probability  of  deferrals  for  assistance. 

e.  Probability  of  deferrals  for  parts. 

f.  Mean  time  of  deferrals  for  assistance. 

g.  Mean  time  of  deferrals  for  parts. 

3.  Model  Runs.  To  test  model  sensitivity  to  changes  in  the  input  variables,  a  single 
input  variable  was  changed  in  each  model  run.  The  change  was  accomplished  using  the 
parametric  variation  ratio  feature  of  the  SHIP-II  model.  Parametric  variation  ratios  can 
be  used  to  uniformly  alter  values  of  selected  variables  by  a  selected  percentage. 

Input  variable  levels  were  changed  in  steps  and  the  results  of  each  model  run 
compared  with  the  previous  baseline  stability  run  using  the  same  random  number  seed.  If 
there  were  significant  differences  between  the  sensitivity  analysis  and  baseline  stability 
runs,  further  runs  were  made  at  a  decreased  level.  This  process  continued  until  there  was 
no  significant  difference  between  the  outputs  of  the  sensitivity  analysis  and  the  baseline 
stability  runs. 

The  steps  chosen  resulted  in  90-,  70-,  50-,  and  10-percent  changes  in  the  input 
variable.  Model  run  cost  considerations  limited  this  analysis  to  these  levels. 
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4.  Analysis  Method.  The  mean  values  of  the  dependent  variables  from  the 
sensitivity  and  baseline  runs  were  compared  using  the  t-test  for  the  difference  between 
two  means.  The  hypothesis  to  be  tested  is  that  two  populations  have  the  same  mean  when 
the  population  standard  deviation,  a,  is  not  known.  To  test  the  hypothesis  that  a  certain 
change  in  the  input  variable  results  in  a  statistically  significant  change  in  the  outputs,  the 
results  of  the  sensitivity  analysis  run  were  compared  with  the  baseline  stability  run. 

a.  HI:  y  j  =  y2,  given  Oj  =  c?2  =  o,  with  the  value  of  o  unknown. 

b.  Choose  a  =  .05. 

c.  For  a  statistic  to  test  this  hypothesis  use 


where  S  ^  is 
P 


S  Sp  Y  (1/Nj)  +  (1/N2> 

the  pooled  mean-square  estimate  of  o  given  by 

-  ((Ex,  )2/Nt)  +  Ex*  - ((Ex2  )2/im2) 

=  _ i _ i _ i _ i _ 

Nj  +  N2 -  2 


9 


where  E 

j  =  sum  of  squares  in  the  baseline  stability  run, 

Ex* 

j  =  sum  of  squares  in  the  sensitivity  analysis  run, 


Ex 


1: 


Ex 


2i 


sum  of  observations  in  the  baseline  stability  run,  and 
sum  of  observations  in  the  sensitivity  analysis  run. 


d.  If  both  populations  have  normal  distributions  with  the  same  mean  and  the 


same  variance,  then  this  statistic  has  the  t 


(Nj  +  N2  -  2) 


distribution. 


test, 


e.  The  critical  region  consists  of  values  of  t  <  t  or  t  >  t  with  a  one-tail 

s  C|  s  q<2 

where  ‘c,  =,Sa(N,  ,N2  -2r  1-796 

*c2  =  'l  -  Sa(Nj  »N2-  2) 

f.  Compute  t$. 

g.  Reject  or  accept  the  hypothesis. 


The  statistically  significant  changes  resulting  from  this  analysis  are  provided  below: 
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a.  Mean  time  between  failures. 

Corrective  Maintenance  Man-hours 


Baseline 

90%  Decrease 

70%  Decrease 

50%  Decrease 

Mean 

61.91 

450.41 

220.64 

107.217 

Std  Deviation 

37.29 

119.75 

57.62 

49.537 

t 

s 

-10.730 

-8.012 

-2.531 

Total  Maintenance  Man-hours 

Mean 

2716.96 

3048.06 

Std  Deviation 

319.36 

302.17 

<s 

-2.613 

b. 

Mean  time  in  function  repair. 

Corrective  Maintenance  Man-hours 

Baseline 

90%  Increase 

50%  Increase 

30%  Increase 

Mean 

61.91 

128.78 

118.518 

124.148 

Std  Deviation 

37.29 

58.17 

68.074 

70.182 

ts 

-3.353 

-2.526 

-2.713 

Division  Training  Man-hours 

Mean 

23.23 

18.730 

Std  Deviation 

5.98 

2.610 

ls 

2.389 

c.  Mean  time  in  function  planned  maintenance. 

Preventive  Maintenance  Man-hours 


Baseline 

90%  Increase 

50%  Increase 

Mean 

203.17 

297.27 

255.134 

Std  Deviation 

50.72 

65.26 

60.838 

*s 

-3.9439 

-2.2726 

Division  Training  Man-hours 

Mean 

23.23 

19.850 

Std  Deviation 

5.98 

1.443 

*s 

-1.90334 
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APPENDIX  C 

SMD/NMP  MANNING  DIFFERENCES 


Tabic  C-l 


SMO/NMP  Manning  RiHcrcncc*  by  Navy  Enlisted 
Classification  (NEC)  Code 


DISTRIBUTION  LIST 


Director  of  Manpower  Analysis  (ODASN(M)) 

Chief  of  Naval  Operations  (OP-01),  (OP- 11),  (OP- 12)  (2),  (OP- 112)  (4),  (OP-112C),  (OP- 
1 15)  (2),  (OP-140F2),  (OP-987H) 

Chief  of  Naval  Material  (NMAT  00),  (NMAT  08L) 

Chief  of  Naval  Research  (Code  200),  (Code  440)  (3),  (Code  442),  (Code  448) 

Chief  of  Information  (01-213) 

Chief  of  Naval  Education  and  Training  (N-5),  (N-9) 

Commander  Fleet  Training  Group,  Pearl  Harbor 
Commander  Naval  Military  Personnel  Command  (NMPC-013C) 

Commanding  Officer,  Naval  Education  and  Training  Program  Development  Center  (Tech¬ 
nical  Library)  (2) 

Commanding  Officer,  Naval  Education  and  Training  Support  Center,  Pacific 
Commanding  Officer,  Naval  Technical  Training  Center,  Corry  Station  (Code  101B) 
Director,  Naval  Civilian  Personnel  Command 
President,  Naval  War  College  (Code  El  14) 

Superintendent,  Naval  Postgraduate  School 

Commander,  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  Alexandria 
(PERI-ASL) 

Chief,  Army  Research  Institute  Field  Unit,  Fort  Harrison 

Commander,  Air  Force  Human  Resources  Laboratory,  Brooks  Air  Force  Base  (Scientific 
and  Technical  Information  Office) 

Commander,  Air  Force  Human  Resources  Laboratory,  Williams  Air  Force  Base 
(AFHRL/OT) 

Commander,  Air  Force  Human  Resources  Laboratory,  Wright-Patterson  Air  Force  Base 
(AFHRL/LR) 

Director,  Plans  and  Programs,  Air  Force  Logistic  Management  Center,  Gunter  Air  Force 
Station 

Superintendent,  U.S.  Coast  Guard  Academy 
Defense  Technical  Information  Center  (DDA)  (12) 


